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SUMMARY 

In  nowadays  complex  and  dynamic  changing  military  environments  supporting  Situation  Awareness  (SA) 
of  operators  is  a  prerequisite  for  situation-  and  task-adequate  decision  making  and  action 
accomplishment.  SA  refers  to  information  on  three  levels  which  represent  relevant  elements  of  the 
operator  environment,  element  patterns  describing  complex  mission  situations,  and  projections  of  future 
states  and  dynamics  of  elements.  A  means  for  supporting  SA  of  operators  are  adaptive  knowledge-based 
user  interfaces.  For  developing  such  interfaces  information  of  the  three  different  SA  levels  which 
operators  need  in  performing  their  tasks  have  to  be  specified  and  modelled.  One  source  from  which  that 
information  can  be  acquired  are  scenarios  which  have  to  be  developed  in  any  case  for  system  design  as 
well  as  for  operator  training.  For  specifying  relevant  SA  information  a  model  of  the  problem  domain  has 
been  developed  which  comprises  the  True  World  of  scenarios,  the  Sensed  World  of  detected  tracks, 
and  the  Deduced  World  of  concluded  track  information.  To  uniformly  describe  these  different  worlds  an 
object  oriented  approach  has  been  applied  which  is  based  on  static  and  dynamic  scenario  and  track 
objects  which  are  specified  mathematically.  Attributes  and  operations  of  track  objects  constitute  elements 
and  patterns  of  relevant  SA  information  to  be  identifed.  Additionally,  the  described  mathematical  model 
of  track  objects  constitutes  the  basis  for  developing  a  software  specification  with  the  object-oriented 
Unified  Modelling  Language  UML.  Using  a  Navy  Anti-Air  Warfare  scenario  as  an  example  the 
application  of  the  developed  modelling  approach  is  demonstrated  in  detail. 


1.0  INTRODUCTION 

Recent  technological  developments  of  sensor  and  weapon  systems  as  well  as  of  all  kinds  of  military 
command,  control,  communication,  and  information  systems  (C2/C3/C4I)  have  increased  the  amount  and 
complexity  of  information  at  hand  while  the  time  available  to  process  that  information  has  dramatically 
decreased  in  present-day  military  operations.  Additionally,  in  actual  military  operations,  e.g.,  in  Eittoral 
Warfare,  Crisis  and  Eow  Intensity  Conflicts,  or  Missions  other  than  War,  operators  who  are  responsible 
for  planning  and  decision  making  are  faced  with  natural  dynamic  situations.  These  situations  can  be 
characterised  by  extremely  rapid  changes  in  the  tactical  situation,  highly  insufficient,  i.e.,  uncertain  and/or 
incomplete  information,  and  a  large  variety  of  potential  situational  hypotheses.  Operators  as  decision 
makers  undergo  high  mental  stress  due  to  the  need  to  respond  quickly  and  accurately,  or  face  potentially 
fatal  consequences.  Human  decision  making  in  such  situations  is  based  on  a  large  scale  on  situation 
awareness  of  operators  which  is  defined  as  the  state  of  operator  knowledge  about  the  external  environment 
resulting  from  situation  perception  and  situation  assessment  (Endsley,  2000). 
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2.0  SITUATION  AWARENESS  AND  ITS  SUPPORT 

Situation  awareness  (SA)  is  the  eonstantly  evolving  degree  of  aeeuraey  by  whieh  operator’s  pereeption 
and  assessment  of  the  external  environment  refleets  reality.  Therefore,  SA  is  the  prerequisite  for  situation- 
and  task-adequate  deeision  making  and  aetion  aeeomplishment  in  eomplex  dynamie  situations  (Fig.  1). 
As  being  the  result  of  a  reliable  assessment  of  the  situation  in  whieh,  e.g.,  a  ship  operates,  SA  is  vital  for 
the  sueeessful  eompletion  of  its  mission.  Therefore,  intending  to  support  military  operators  in  deeision 
making  and  aetion  taking  in  any  ease  and  first  of  all  SA  has  to  be  supported.  Aeeording  to  Endsley, 
operator’s  SA  is  the  perception  of  elements  in  his/her  environment  within  a  volume  of  time  and  space, 
the  comprehension  of  their  meaning,  and  the  projection  of  their  state  in  the  near  future.  To  specify 
required  information  Endsley  (1995)  distinguishes  three  levels  of  SA:  1.)  Perception  of  the  information 
elements  in  the  environment,  i.e.,  the  states,  attributes,  and  dynamics  of  relevant  elements  in  the  operator 
environment.  2.)  Comprehension  of  the  current  situation  by  information  processing  based  on  a  synthesis  of 
disjoint  level  1  elements  by  putting  them  together  to  perform  patterns  for  getting  a  holistic  picture  of  the 
environment  and  an  assessment  of  the  current  state  3.)  Projection  of  future  states  on  the  basis  of  actions  of 
elements  in  the  environment.  This  is  achieved  through  knowledge  of  the  states  and  dynamics  of  the 
elements  and  comprehension  of  the  situation  for  both  level  1  and  level  2. 


Actual  information 

about 

the  external  system  environment  and  the  internal  system  state 


the  external  system  environment  and  internal  system  states  complexity 

Figure  1:  Structure  of  Human  Problem  Solving  Activities. 


To  establish  SA,  information  about  the  environmental  situation  from  own  sensors  or  other  sources  of 
significant  data  and  conditions  must  be  gathered  and  processed.  This  information  concerns  the  external 
system  environment  as  well  as  the  internal  operational  state(s)  of  own  combat  (sub)system(s)  involved. 
But  not  only  actual  situation  information  is  necessary  for  accomplishing  the  process  to  reach  SA. 
For  Eevel  2  and  Eevel  3  of  SA  which  contain  assessment  steps  pre-existing  relevant  “a  priori”  information 
is  necessary  to  relate  actual  information  elements  to  already  known  situational  information  and  patterns. 
The  following  considerations  will  be  focussed  especially  on  a  priori  information  about  the  external  system 
environment.  SA  occurs  as  a  consequence  of  integrating  a  priori  information  together  with  actual 
information  by  cognitive  processing  skills  that  include  attention  allocation,  perception,  data  extraction, 
comprehension,  and  projection  (Salas  et  ah,  1995).  In  order  to  provide  a  reliable  information  basis  for 
carrying  out  missions  it  will  be  necessary  to  assess  and  reassess  the  situation  on  a  continuous  basis. 
Therefore,  to  not  only  establish  but  also  to  maintain  SA  the  continuous  extraction  of  information  about  a 
dynamic  system  and/or  environment,  the  integration  of  this  information  with  previously  acquired 
knowledge  to  form  a  coherent  mental  picture,  and  the  use  of  that  picture  in  directing  further  perception  of, 
anticipation  of,  and  attention  to  future  events  is  necessary  (Wickens,  1996). 
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Operator  support  by  intelligent  and  adaptive  knowledge-based  user  interfaces  is  considered  to  be  a  viable 
approach  to  overcome  some  of  those  difficulties  decision  makers  are  faced  with  when  having  to  cope  with 
complex  command  and  control  systems  in  novel  military  situations.  It  may  currently  not  be  possible  to 
design  a  system  which  can  cope  with  all  conceivable  events  in  highly  ambiguous  situations,  e.g.,  with 
those  found  in  present-day  military  operations.  But  it  is  already  possible  to  develop  systems  that 
complements  human’s  abilities  in  perceiving  and  assessing  such  situations  as  well  as  appropriately 
responding  in  unknown  situations.  Such  intelligent  user  interfaces  consist  of  a  knowledge-based  assistance 
system  and  an  interactive  graphical  or  multimedia  user  interface  (Fig.  2).  They  can  support  military 
decision  makers  in  performing  information  gathering,  information  processing,  and  information  entering  in 
all  phases  of  a  command  and  control  (C2)  cycle,  i.e.,  in  situation  perception  (observe),  situation 
assessment  (orient),  decision  making  (decide),  and  action  taking  (act)  (Dept,  of  the  Navy,  1995).  The  basic 
idea  of  these  concepts  is  that  an  overall  automation  must  not  be  the  objective  of  system  development. 
The  human  operator  should  be  involved  in  the  decision  making  process  as  far  as  his  abilities  and 
his  performance  are  sufficient  for  goal  achievement.  An  aid  is  provided  only  to  exploit  human  abilities 
(e.g.  in  detecting  and  evaluating  complex  patterns  or  reacting  on  unforeseen  events)  and  to  overcome 
human  deficiencies  (e.g.  when  doing  mathematical  calculations),  i.e.,  to  complement  individual  human 
performances. 


External  System  Environment 

Figure  2:  Concept  of  a  Knowledge-Based  User  Interface. 

For  the  design  of  effective  human-machine  interfaces  which  support  human  operators  in  military 
operations  required  SA  information  has  to  be  determined,  specified,  and  implemented.  One  source  from 
which  especially  information  about  the  external  system  environment  can  be  acquired  are  scenarios  which 
have  to  be  developed  for  designing  systems  as  well  as  for  training  military  operators.  In  a  recent  study 
(Distelmaier  et  al.,  2000)  this  acquisition  process  has  been  accomplished  on  the  basis  of  an  Anti-Air 
Warfare  scenario  developed  for  training  Navy  operators  in  identifying  air  targets  in  a  surveillance  mission. 

The  scenario  includes  ownship  with  different  safety  and  engagement  zones,  an  airway,  a  transition 
corridor,  a  land  area  with  coastal  line,  neutral  and  friendly  air  targets  with  normal  behaviour,  and  suspect 
air  targets  with  dubious  behaviour.  The  scenario  describes  graphically  not  only  a  snapshot  of  a  dynamic 
situation  but  rather  a  combination  of  different  static  scenes  of  a  dynamically  evolving  situation  with  air 
tracks  to  be  identified.  It  includes  the  following  situations: 

1 .  An  approach  of  two  air  targets  with  suspicious  behaviour. 

2.  An  approach  of  two  friendly  air  targets  which  identify  themselves  by  executing  a  predefined  flight 
pattern. 
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3.  An  approach  of  a  friendly  air  target  in  a  transition  corridor. 

4.  Neutral  air  targets  flying  in  an  airway. 

5.  An  approach  of  a  suspicious  air  target  from  inside  the  airway  with  a  final  harassment  manoeuvre. 


3.0  MODEL  OF  THE  PROBLEM  DOMAIN 

To  acquire  the  information  necessary  for  getting  SA  a  model  of  the  scenario  problem  domain  has  been 
developed.  The  model  comprises  three  worlds  (Fig.  3)  which  represent  three  different  views:  1.)  The  True 
World  stands  for  the  real  mission  environment  of  ownship  respectively  for  the  developed  scenario  which 
is  a  model  of  that  environment.  2.)  The  Sensed  World  describes  the  information  sphere  of  track  objects 
acquired  to  a  large  extent  by  ownship  sensors.  3.)  The  Deduced  World  represents  the  sphere  of  deduced 
information  concluded  from  the  Sensed  World  and  corresponding  inference  processes.  To  uniformly 
describe  these  different  views  an  object  oriented  approach  is  applied.  This  approach  also  supports  an 
object  oriented  problem  analysis  and  facilitates  the  implementation  with  modem  object-oriented 
programming  languages  when  later  developing  a  decision  support  system.  To  accomplish  this  approach 
static  and  dynamic  objects  are  identified  as  relevant  model  elements.  These  objects  can  be  specified  by 
means  of  attributes  which  describe  characteristics  and  states  of  an  object  and  by  operations  which 
characterise  its  behaviour  (Rumbaugh  et  al.,  1991). 


3,1  The  True  World 

Descriptive  elements  of  the  Tme  World  which  is  defined  by  the  used  scenario  are  static  and  dynamic 
scenario  objects.  Static  scenario  objects  possess  only  one  state  and  no  operation.  Examples  of  such  objects 
which  constitute  the  static  mission  environment  of  the  scenario  are,  for  instance,  airways,  transition 
corridors,  and  coastal  lines.  Dynamic  scenario  objects  correspond  to  active  air,  surface,  and  land  targets  of 
the  scenario.  They  are  specified  by  changing  attribute  values,  that  means,  changing  object  states  caused  by 
object  operations. 


•  Real  operational 
environment. 

•  Scenarios  described  by 
static  and  dynamic 
scenario  objects 
representing  the  real 
operational  environment. 


•  Static  track  objects  with 
invariable  attributes. 

•  Dynamic  track  objects 
with  sensed  variable 
attributes  and  operations. 


•  Dynamic  track  objects 
with  deduced  attributes 
and  operations  which 
represent  inference 
processes. 


Figure  3:  Model  of  the  Three  Worlds. 


To  describe  scenario  objects  mathematically  formalisms  of  the  general  system  theory  (e.g.,  Klir,  1969) 
have  been  applied.  Generally,  the  analysis  starts  with  determining  the  set  SO  of  all  scenario  objects  soi : 

SO  =  {  so, :  i  e  Iso}  .  (1) 
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An  scenario  object  soi  can  be  described  by  the  set  sOi_OP  of  its  operations  soj  opk,  the  set  sOi_ATT  of 
its  attributes  S0i_attk,  the  value  set  sOi  ATVk  of  all  values  S0i_attk(t)  of  an  attribute  S0i_attk,  with  the  time 
set  T,  the  set  of  the  positive  real  numbers  R+,  and  t  e  T  c  R+  u  {  0  },  and  the  set  sOi_S  of  object  states 
so,_Sk : 

sOi  OP  =  {  so,_opk  :  k  s  hoi  op  }  , 

so._ATT  =  {  so,_attk  :  k  e  hoi  att  }  , 

so,_ATVk  =  {  S0i_attk(t) :  t  e  T  }  ,  (2) 

S0i_S=  {  S0i_Sk  =  (sOi_attin(t),  sOi_atWi(t),  S0i_attin+2(t),  ... )  :  k  e  hoys  a  so,_attm(t)  e 
S0i_ATVin  A  so,_atWi(t)  e  so,_ATVni+i  a  sOi_atW2(t)  e  S0i_ATVm+2  a  ...  }  , 

SO,_S  CZ  SOi_ATVmX  SOi_ATVni+i  X  SOi_ATVm+2  X . 


As  an  example,  the  airway  identified  in  the  mentioned  Navy  scenario  is  considered.  It  represents  a  static 
scenario  object  soi  which  is  characterised  by  its  attributes  soi_attk  and  their  related  values  soi_attk(t). 
Because  Oi  represents  a  static  scenario  object  there  exists  no  operation  and  all  related  attribute  values  are 
constant: 

soi_object_identifier(t)  =  airway  number , 

soi_reference_point(t)  =  (posX(t):  a  [°],  posY(t):  b  [°],  posZ(t):  c  [ft]) , 
soi_  width(t)  =  d  [nm] , 
soi_  length(t)  =  e  [nm] , 

soi_  height(t)  =  f  [ft] ,  (3) 

soi_  direction(t)  =  g  [°] , 

soi_region(t)  =  function(soi_reference_point(t),  soi_width(t),  soi_length(t),  soi_height(t), 
soi_direction(t))  , 

soi_  speed(t)  =  h  [kn] , 

soi_  flight_level(t)  =  (i,  j)  [ft] . 

As  another  example  of  the  scenario  an  aircraft  inside  the  airway  is  considered.  It  appears  to  be  a  dynamic 
scenario  object  SO4  which  can  be  specified  by  attributes,  their  values,  and  operations.  Some  attribute  values 
S04_attk(t)  and  operations  so4_opk  are: 

S04_aircraft_identifier(t)  =  registration  number  , 
so4_position(t)  =  (posX(t):  x  [°],  posY(t):  y  [°],  posZ(t) :  z  [ft]) , 

S04_altitude(t)  =  so4_posZ(t)  =  z  [ft] , 

S04_altitude_change(t)  =  0  [ft/min] , 

so4_course(t)  =  g  [°] ,  (4) 

so4_course_change(t)  =  0  [7sec] , 
so4_speed(t)  =  h  [kn] , 
so4_speed_change(t)  =  0  [kn/min] , 

S04_IFF_signal(t)  =  Mode  3  , 

S04_emitter(t)  =  Radar  R3  , 
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so4_role(t)  =  commercial  airliner , 

S04_activity(t)  =  fly  in  accordance  with  airway , 

S04_identity(t)  =  neutral . 

so4_opi  =  IF  creation  event  TFIEN  create  object , 

so4_op2  =  IF  extinction  event  TFIEN  delete  object , 

so4_op3  =  IF  state  change  event  THEN  change  state  EESE  retain  state  . 

The  dynamic  scenario  object  04  may  be  in  the  actual  state  S04_si  =  flying  inside  the  airway  which  is 
defined  by  the  above  attributes.  As  soon  as  there  occurs  an  event,  for  instance,  if  the  value 
so4_course_change(t)  >  x  [7sec]  indicates  a  course  change  then  the  activity  takes  the  value  S04_att2(t)  = 
fly  not  in  accordance  with  airway.  The  state  remains  the  same. 

The  dynamic  processes  of  a  scenario  with  all  state  changes  of  scenario  objects  can  be  simulated  and  in  this 
way  be  accessible  to  an  analysis.  One  possible  simulation  tool  is,  e.g.,  the  Commercial-Off-The-Shelf 
(COTS)  product  STAGE  (Scenario  Toolkit  And  Generation  Environment).  STAGE  is  a  real-time, 
reconfigurable,  extendible  simulation  framework  for  military  applications.  It  is  a  complete  toolkit  not  only 
for  tactical  simulation  but  also  for  research  and  development,  man-in-the-loop  simulation,  and  mission 
planning  and  rehearsal  (Virtual  Prototypes,  2002). 

3.2  The  Sensed  World 

The  second  part  of  the  model  constitutes  the  Sensed  World  (Fig.  3)  which  describes  the  information  sphere 
of  track  objects  acquired  to  a  large  extent  by  ownship  sensors.  Corresponding  to  scenario  objects  of  the 
True  World  there  are  again  static  and  dynamic  track  objects  in  the  Sensed  World  with  the  same  meaning 
as  the  scenario  objects.  Static  track  objects  in  the  Sensed  World  correspond  for  the  most  part  to  static 
scenario  objects  and,  therefore,  are  known  in  advance  either  from  the  scenario  or  other  geographical  data 
sources  like  nautical  charts.  Examples  of  such  static  track  objects  which  posses  only  one  state  and  no 
operation  are  again  airways,  transition  corridors,  and  coastal  lines  constituting  the  static  mission 
environment.  But  there  may  be  certain  static  track  objects  which  ship  sensors  may  detect,  e.g.,  the  wreck 
of  a  recently  sunk  ship  which  may  be  detected  by  the  ship  sonar  but  not  being  registered  yet  in  the 
corresponding  nautical  chart.  All  static  objects  are  stored  onboard  ownship,  for  instance,  in  the  central  data 
store  of  the  ship  which  may  contain  also  a  geographical  database  with  nautical  chart  information.  Dynamic 
track  objects  correspond  again  to  dynamic  scenario  objects.  They  represent  active  air,  surface,  and  land 
tracks  detected  by  ownship  sensors  and  stored  with  attributes  and  operations  in  the  central  data  store. 
That  means,  that  dynamic  scenario  objects  of  the  True  World  are  transformed  into  track  objects  of  the 
Sensed  World  by  considering  ship  sensor  characteristics.  Attributes  and  their  values  are  updated  if  sensors 
provide  new  data.  As  attributes  of  dynamic  track  objects  depend  on  the  available  sensors  on  board, 
only  that  information  can  be  sensed  for  which  sensors  are  available.  For  instance,  if  there  is  a  3D-radar 
available  then  the  altitude  of  an  air  track  can  be  determined  as  track  attribute.  If  the  ship  has  only  a 
2D-radar  then  the  altitude  cannot  be  assessed.  But  there  are  also  track  attributes  which  can  be  determined 
from  sensed  attributes  by  calculation,  e.g.,  the  vertical  speed  of  an  air  track  from  the  change  of  its  altitude. 
These  attributes  are  also  considered  as  sensed  attributes.  With  dynamic  track  objects  operations  specify 
processes  like  creating,  updating,  and  deleting  those  objects  in  the  data  store  of  the  ship. 

With  TO  as  the  set  of  all  track  objects  tOj  again  such  an  object  can  be  described  formally  by  a  set  toi  OP 
of  its  operations  tOi  Opk,  a  set  tOi  ATT  of  its  attributes  tOi_attk,  a  value  set  tOi_ATVk  of  all  values  tOi_attk(t) 
of  attribute  tOi_attk,  with  the  time  set  T,  the  set  of  positive  real  numbers  R+,  and  t  e  T  c  R+  u  {  0  } ,  a  set 
tOi_ATV  of  all  value  sets  tOi  ATVk,  and  a  set  tOi_S  of  all  states  tOi_Sk  of  object  tOj: 

TO  =  {  to. :  i  s  Ito  }  , 
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to,  OP  =  { to.  opk.  :  k  e  koi  op  }  , 
to,_ATT  =  {  to,_attk  :  k  s  koi  att  }  , 

to,_ATVk  =  { tOi_attk(t)  :  t  e  T  }  ,  (5) 

to,_ATV  =  {  to,_ATVk  :  k  e  koi_ATT }  , 

to,_S  cz  tOi_ATVkX  tOi_ATVk+i  x  tOi_ATVk+2  x  ....  , 

tOi_S  =  { tOi_Sni  =  (tOi_attk(t),  tOi_attk+i(t),  tOi_attk+2(t),  ... )  :  m  e  koi  s  ^  tOi_attk(t)  e 
tOi_ATVk  A  tOi_attk+i(t)  e  tOi_ATVk+i  a  tOi_attk+2(t)  e  tOi_ATVk+2  a  ...  }  . 

As  an  example  of  a  static  track  object  the  above  mentioned  airway  which  represents  in  the  True  World  the 
static  scenario  object  soi  is  considered.  In  the  Sensed  World  it  constitutes  the  static  track  object  toi  with 
the  same  attributes  and  values  as  soi  specified  in  equation  (3).  As  example  of  a  dynamic  track  object  the 
above  mentioned  aircraft  is  regarded.  In  the  True  World  this  aircraft  has  been  represented  by  the  dynamic 
scenario  object  SO4.  If  this  object  is  in  the  ownship  sensor  range  it  will  be  detected  and  a  dynamic  track 
object,  e.g.,  to?  will  be  created  in  the  ship’s  central  data  store  which  represents  the  Sensed  World.  Object 
attributes  and  values  depend  on  the  sensor  observation  time.  It  is  assumed  that  attributes  like  altitude, 
speed,  and  course  and  their  alteration  can  be  determined.  Assuming  sufficient  observation  time  for 
reaching  a  stable  state  values  t07_attk(t)  of  some  sensed  attributes  t07_attk  and  operations  t07_opk  are  listed 
below: 

t07_track_identifier(t)  =  track  number  , 

t07_position  =  ( posX(t):  x  [°] ,  posY(t):  y  [°] ,  posZ(t):  z  [ft] ) , 
t07_altitude(t)  =  t07_posZ(t)  »  z  [ft] , 
t07_altitude_change(t)  »  0  [ft/min] , 

t07_course(t)  ~  g  [°] ,  (6) 

t07_course_change(t)  »  0  [7sec] , 

t07_speed(t)  »  h  [kn] , 

t07_speed_change(t)  »  0  [kn/min] , 

t07_IFF_signal(t)  =  Mode  3  , 

t07_emitter(t)  =  Radar  R3  , 

t07_opi  =  IF  detection  event  THEN  create  object , 
t07_op2  =  IF  update  event  THEN  update  sensed  attributes  , 
t07_op3  =  IF  extinction  event  THEN  delete  object . 

3,3  The  Deduced  World 

The  Deduced  World  is  the  third  part  of  the  developed  model  (Fig.  3).  It  is  represented  by  the  deduced 
attributes  of  dynamic  track  objects,  their  values,  and  by  the  inference  processes  necessary  for  deducing 
those  attributes.  Taking  again  as  an  example  the  dynamic  track  object  t07  specified  above  the  following 
additional  deduced  attributes  t07_d-attk  and  operations  t07_d-opk  of  the  object  t07  arise  in  the  Deduced 
World: 

t07_d-distance_between_objects,  t07_d-formation, 

t07_d-activity,  t07_d-activity  sequence. 
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toyd-role, 

t07_d-category, 

t07_d-class, 

t07_d-identity, 


t07_d-application, 

t07_d-type, 

t07_d-option, 

t07_d-threat. 


t07_d-op4  =  IF  update  event  THEN  update  dedueed  attributes 


(7) 


Generally,  with  equations  (5)  for  a  dedueed  attribute  tOi_d-attm  of  traek  object  tOi,  its  values  tOi_d-attni(t) 
and  the  object  operation  tOi_d-opk  which  specifies  the  update  and  inference  process  of  these  attribute  it 
counts: 

to,_d-attm  e  tOi_ATT , 

to,_d-attni(t)  e  tOi_d-ATVm  e  tOi_  ATV  ,  (8) 

to,_d-opk  e  tOi  OP  . 


A  deduced  attribute  of  a  dynamic  track  object  can  be  derived  from  sensed  and  other  already  deduced 
attributes  of  the  same  object.  Additionally,  different  attributes  of  other  static  and  dynamic  track 
objects  may  be  involved  in  the  inference  process  as  well.  For  modelling  that  interference  process 
mathematical  relations  are  applied.  To  deduce  an  attribute  tOi_d-attm  of  a  dynamic  track  object  tOi  and  its 
value  set  tOi  d-ATVm  those  other  value  sets  tOi_ATVn  and  tOi_d-ATVn  of  tOi  which  contribute  to  the 
inference  process  have  to  be  selected.  For  describing  this  selection  a  set  tOj  ATVsel  which  contains  all 
contributing  value  sets  tOi  ATVn  and  tOi_d-ATVn  will  be  defined.  Besides  tOi  there  may  be  other  track 
objects  tOp,  tOq,  ...  with  their  attribute  value  sets  tOp  ATV,  tOq  ATV  ...  contributing  as  well  for  deducing 
the  attribute  tOi  d-attm.  The  contributing  value  sets  of  those  objects  can  be  again  specified  by  means  of 
selected  value  sets  tOp  ATVsel,  tOq  ATVsel  ....  Then,  to  describe  the  inference  process  in  detail  an 
inference  relation  tOj  irkis  defined.  With  tOj  IR  as  the  set  of  all  tOi  hk  it  follows: 

to,_ATVsel  =  { to,_ATVn,  tOi_d-ATVn+k  :  tOi_ATVn,  tOi_d-ATVn+k  e  tOi_ATV  a  tOi_ATVn  and 
tOi_d-attn+k  are  relevant  to  deduce  tOi_d-attn, }  , 

tOp_ATVsel  =  { tOp_ATVu,  tOp_d-ATVu+k  :  tOp_ATVu,  tOp_d-ATVu+k  e  tOp_ATV  a  tOp_ATVu  and 
tOp_d-attu+k  are  relevant  to  deduce  tOi  d-attm  }  , 

tOq_ATVsel  =  { tOq_ATVv,  tOq_d-ATVv+k  :  tOq_ATVv,  tOq_d-ATVv+k  e  tOq_ATV  a  tOq_ATVy  and 
tOq_d-attv+k  are  relevant  to  deduce  tOi  d-attm  }  , 

to,_ATVsel  cz  tOi_ATV  ,  tOp_ATVsel  cz  tOp_ATV  ,  tOq_ATVsel  cz  tOq_ATV  , 

to,  irk  tOi  d-ATVm  x  X  tOi  ATVsel  x  X  tOp  ATVsel  x  X  tOq  ATVsel  x  ....,  (9) 

to,  JR  =  {  tOi_irk :  k  e  Toi  ir  }  , 

IR  =  {  to,  IR :  i  s  Ito  }  • 


The  set  to  JR  containing  all  relations  of  a  dynamic  track  object  to,  specifies  the  total  inference  process  of 
that  object.  This  process  is  part  of  the  object  operation  tOi_d-op4  =  “IF  update  event  THEN  update  deduced 
attributes”  and  will  be  activated  if  an  update  event  occurs.  Moreover,  the  total  Deduced  World  is 
represented  by  the  set  of  all  relations  IR  which  comprises  total  inference  processes  of  all  considered 
dynamic  track  objects.  This  inference  process  represents  especially  the  a  priori  knowledge  needed  by 
human  operators  when  performing  their  operational  tasks. 

As  an  example,  again  the  dynamic  track  object  t07  representing  an  aircraft  is  selected.  To  specify  the 
inference  process  for  deducing  the  attribute  t07_d-activity  the  value  set  t07_d-ACTIVITY  is  interrelated 
with  value  sets  of  relevant  sensed  attributes  t07  (POSX,  POSY,  POSZ),  t07  AEITUDE,  t07  COURSE, 


10-8 


RTO-MP-088 


Modelling  Situation  Awareness  Information  for  Naval  Decision  Support  Design 


and  tov  SPEED  of  object  tov  (see  Eq.  6).  (In  the  preceding  and  in  the  following  all  value  sets  of  an  attribute 
are  named  with  capital  letters.)  Additionally,  value  sets  of  the  static  track  object  toi  representing  an  airway 
have  to  be  considered  in  this  inference  process.  Airway  attribute  value  sets  are  toi  REGION, 
toi  DIRECTION,  toi  SPEED,  and  toi  FEIGHT  EEVEE  (see  Eq.  3).  If  the  process  is  specified  with 
equations  (9)  then  the  inference  relation  07  hi  results  as  follows: 

t07  ATVsel  =  { t07_(POSX,  POSY,  POSZ),  t07_AETITUDE,  t07_COURSE,  t07_SPEED  }  , 

toi  ATVsel  =  { to  1  REGION,  to  i  DIRECTION,  to  1  SPEED,  toi  FEIGHT  EEVEE  }  ,  (10) 

t07_iri  a  t07_d-ACTIVITY  x  X  t07  ATVsel  x  X  to  1  ATVsel . 

For  representing  graphically  this  inference  process  interaction  matrices  introduced  by  Sage  (1991)  can  be 
applied.  Figure  4  shows  the  example  described  with  equations  (10).  In  the  upper  part  of  the  picture  the 
static  track  object  toi  with  its  attribute  value  sets  is  displayed.  The  lower  part  shows  attribute  value  sets  of 
the  dynamic  track  object  of  interest  t07.  The  arrows  indicate  the  direction  of  the  inference  process. 
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Figure  4:  Example  of  the  Inference  Process  for  the  Deduced  Attribute  t07_d-activity. 


For  describing  the  inference  process  in  detail  a  table  form  can  be  used.  As  an  example,  in  Table  1  some 
components  of  the  relation  t07_iri  which  specifies  the  attribute  values  for  deducing  the  attribute 
t07_d-activity  with  reference  to  the  airway  are  listed.  Each  row  of  the  table  specifies  a  single  component  of 
that  relation  whereby  attribute  values  of  the  airway  are  taken  from  equation  (3).  Of  course  there  are  still 
other  values  t07_d-activity(t)  of  that  attribute  which  arise  by  referencing  the  dynamic  track  object  to? 
to  other  objects.  For  instance,  by  additionally  referencing  to?  to  ownship  the  resulting  value  set  is 
t07_d-ACTIVITY  =  {  fly  in  accordance  with  airway,  fly  not  in  accordance  with  airway,  pass  ownship, 
fly  ownship  inbound  manoeuvre,  leave  ownship  } . 

For  designing  a  decision  support  system  the  described  mathematical  model  of  track  objects  constitutes  the 
basis  for  developing  a  software  specification  with  the  object-oriented  Unified  Modelling  Eanguage  UME 
(Booch  et  al.,  1999).  This  specification  may  comprise  different  types  of  diagrams  like  class  diagrams, 
state  diagrams,  sequence  diagrams,  etc.  As  an  example,  figure  5  shows  upper  levels  of  a  class  diagram  of 
the  modelled  track  objects  containing  the  class  Track  Objects  with  its  subclasses  Static  Track 
Objects,  Dynamic  Track  Objects,  and  Inference  Processes.  In  this  diagram  inference  processes  are  dealt 
with  as  separate  association  class  which  is  used  usually  to  specify  complex  association  between  other 
classes.  Each  subclass  can  be  further  decomposed  into  more  elementary  classes  with  smaller  dimensions, 
for  instance,  the  class  Dynamic  Track  Objects  into  classes  like  Air,  Surface,  and  Subsurface,  and  Eand 
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Tracks.  Nowadays,  the  development  of  an  UML  specification  will  be  supported  by  modelling  tools 
like  TOGETHER  (TogetherSoft  Corp.,  2002)  which  simplifies  and  integrates  the  analysis,  design, 
implementation,  deployment,  and  debugging  of  complex  software  applications. 


Table  1:  Example  Components  of  the  Relation  toyjri  (Eq.  10) 
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4.0  RELATIONSHIPS  BETWEEN  THREE  WORLDS  AND  INFORMATION 
LEVELS  OF  SA 

To  establish  relationships  between  the  three  worlds  and  the  information  levels  of  SA  described  in  the 
beginning  the  specified  objects  and  their  attributes  are  analysed  in  detail.  Because  an  operator  in  the 
combat  information  centre  of  a  warship  does  not  have  a  direct  contact  to  the  mission  environment  outside 
the  ship  it  becomes  obviously  that  there  does  not  exist  any  direct  relationship  between  the  True  World  and 
the  different  SA  level  information.  The  True  World  represents  either  the  real  mission  environment  of  a 
ship  or  a  scenario  as  a  model  of  this  environment.  An  operator,  for  instance,  in  the  ownship  combat 
direction  centre  does  not  have  any  direct  access  to  this  world  but  there  is  only  an  indirect  interaction 
between  both  via  sensors  and  communication  facilities  on  board  the  ship.  Nevertheless,  identified  static 
scenario  objects,  like  air  routes,  transition  corridors,  and  coastal  lines,  as  well  as  dynamic  scenario  objects, 
like  air  and  surface  targets,  constitute  the  starting  point  of  the  analysis  because  from  them  objects  and  their 
attributes  of  the  Sensed  World  can  be  derived. 

The  Sensed  World  represent  the  basis  for  identifying  elements  of  the  SA  level  1  which  refers  to  the 
perception  of  information  elements  in  the  environment  of  an  operator.  This  environment  is  represented  by 
characteristics  of  track  objects,  i.e.,  their  attributes,  states,  and  behaviour.  Therefore,  relevant  SA 
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information  are  attributes  of  static  track  objects  which,  as  described  in  detail  above,  correspond  to 
attributes  of  static  scenario  objects  of  the  True  World.  This  information  represents  a  priori  knowledge 
stored  on  board,  for  instance,  in  a  geographical  data  base.  Other  information  of  SA  level  1  are  attributes  of 
dynamic  track  objects  which  can  be  identified  by  considering  available  sensors  and  communication 
facilities  on  board  the  ship  and  present  dynamic  scenario  objects  of  the  True  World.  Such  track  attributes 
are,  e.g.,  position,  course,  and  speed.  This  SA  information  is  stored  as  dynamic  track  objects  with  their 
attributes,  e.g.,  in  the  central  data  store  of  the  ship  and  displayed  in  any  form  on  consoles  of  the  combat 
information  centre. 


Figure  5:  UML  Class  Diagram  for  Upper  Levels  of  Track  Object  Classes. 

The  Deduced  World  contains  the  same  dynamic  track  objects  as  the  Sensed  World  but  with  additionally 
deduced  attributes.  These  additional  attributes  constitute  information  of  SA  level  2  which  are  necessary  for 
operator’s  comprehension  of  the  current  situation.  By  information  processing  based  on  a  synthesis  of 
disjoint  SA  level  1  elements  the  operator  puts  these  elements  together  to  perform  patterns  for  getting  a 
holistic  picture  of  the  environment  and  an  assessment  of  the  current  state.  As  an  example,  again  the 
dynamic  track  object  tov  which  represents  an  aircraft  can  be  considered.  Deduced  attributes  of  this  object 
belonging  to  SA  level  2  information  are,  e.g.,  t07_d-activity,  t07_d-type,  and  t07_d-identity. 

But  in  addition,  the  Deduced  World  contains  also  attributes  which  reflect  information  of  SA  level  3. 
This  information  represents  projections  of  future  states  of  a  dynamic  track  object  on  the  basis  of  its  actual 
actions  and  possible  actions  in  future.  This  is  achieved  through  knowledge  of  object  states  and  behaviour 
and  comprehension  of  the  situation  for  both  level  1  and  level  2  information.  As  an  example  of  such  a 
deduced  attribute  belonging  to  SA  level  3,  e.g.,  the  attribute  t07_d-option  of  the  air  track  t07  is  considered. 
In  contrary  to  the  deduced  attribute  t07_d-activity  which  describes  the  actual  observable  behaviour  of  the 
air  track  the  attribute  t07_d-option  portrays  the  predictable  possible  object  behaviour  in  the  near  future 
(with  a  maximal  prediction  time  of  about  five  minutes  ahead).  Both  attributes  posses  the  same  value  sets 
(see  Table  2).  Another  deduced  attribute  which  belongs  to  the  SA  level  3  is  the  attribute  t07_d-threat  with 
a  value  set  t07_d-THREAT.  This  attribute  describes  possible  future  threats  which  depend  on  the  actual 
activity  of  the  air  track  and  its  possible  future  options.  The  inference  process  of  this  attribute  can  be 
specified  with  the  relation  t07_ir2  as  follows: 

t07_d-THREAT  =  {  no  threat,  looming  threat,  acute  threat,  critical  threat }  , 

t07  ATVsel  =  { t07_d-ACTIVITY,  07_d-OPTION  }  ,  (11) 

t07_ir2  c=  t07_d-THREAT  x  t07_d-ACTIVITY  x  Oi_d-OPTION  . 
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Some  components  of  this  relation  are  specified  in  table  2.  The  first  column  of  the  table  does  not  belong  to 
the  relation  toy  ira  but  contains  advancing  points  in  time  to  demonstrate  a  possible  dynamic  progress  of  the 
situation  in  time.  For  each  point  in  time  different  components  of  the  relation  exist  indicating  that  each 
activity  has  more  than  one  possible  option  in  future.  E.g.,  if  the  air  track  does  not  fly  in  accordance  with 
the  airway  then  possible  options,  i.e.,  future  activities,  are  to  leave  ownship,  to  pass  ownship  outside  its 
closest  point  of  approach  (CPA),  or  to  fly  an  ownship  inbound  manoeuvre,  i.e.,  approaching  inside 
ownship  CPA.  The  related  threat  values  depending  on  the  combination  of  activity  and  option  values  at 
each  time  (see  Table  2)  are  expressed  on  a  ordinal  scale. 


Table  2:  Example  Components  of  the  Relation  t07_ir2(Eq.  11) 
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fly  ownship  inbound  manoeuvre 

fm+2 

acute  threat 

fly  ownship  inbound  manoeuvre 

reconnoitre  ownship 

acute  threat 

fly  ownship  inbound  manoeuvre 

harass  ownship 

acute  threat 

fly  ownship  inbound  manoeuvre 

release  weapon 

fm+3 

critical  threat 

release  weapon 

attack  ownship 

5.0  CONCLUSION 

In  nowadays  complex  and  dynamic  changing  military  environments  supporting  Situation  Awareness  (SA) 
of  operators  is  a  prerequisite  for  situation  and  tasks  adequate  decision  making  and  action  accomplishment. 
A  means  for  supporting  SA  of  operators  are  adaptive  knowledge-based  user  interfaces.  For  developing 
such  interfaces  information  of  Endsley’s  three  different  SA  levels  which  operators  need  in  performing 
their  tasks  have  to  be  specified  and  modelled.  One  source  from  which  that  information  can  be  acquired 
are  scenarios  which  have  to  be  developed  in  any  case  for  system  design  as  well  as  for  operator  training. 
For  identifying  relevant  SA  information  of  operators  a  model  of  the  scenario  problem  domain  has  been 
developed  which  comprises  the  True  World,  the  Sensed  World,  and  the  Deduced  World.  To  uniformly 
describe  these  different  worlds  an  object-oriented  approach  has  been  applied  which  is  based  on  static  and 
dynamic  scenario  and  track  objects  which  are  specified  mathematically.  Attributes  and  operations  of  track 
objects  constitute  elements  and  patterns  of  relevant  SA  information  to  be  identified.  Main  object 
characteristics  are  attributes  and  operations.  Interrelating  this  characteristics  with  information  of  different 
SA  levels  the  relevant  SA  information  needed  by  operators  can  be  identified  and  modelled  exhaustively 
and  clearly.  Additionally,  the  described  mathematical  model  of  track  objects  constitutes  the  basis 
for  developing  a  software  specification  with  the  object-oriented  Unified  Modelling  Eanguage  UME. 
Therefore,  the  described  modelling  approach  supports  an  object  oriented  problem  analysis  and  facilitates 
the  implementation  with  modem  object-oriented  programming  languages  when  later  developing  a  decision 
support  system.  The  application  of  the  approach  and  its  advantageous  stmcture  have  been  demonstrated  in 
detail  using  as  an  example  a  Navy  Anti-Air  Warfare  scenario  developed  for  training  Navy  operators  in 
identifying  air  targets  in  a  surveillance  mission. 
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